Post

Replies

Boosts

Views

Activity

Ten FSKit issues found building a network file system module (all filed with minimal repros)
While building an SMB 2/3 client as an FSKit file system module (FSUnaryFileSystem + FSVolume, the macOS 27 Handler protocols), I ran into a number of framework-level issues. I have filed each one with a title starting "FSKit:" so they are easy to find, and every report has a minimal reproduction attached: a small in-memory FSKit module (no network, no disk, no cache of its own), so none of them depend on SMB. All were measured on macOS 27.0 (26A5406e and 26A5416b) with Xcode 27.0 beta 5. Summaries below in case anyone else is hitting these. FB24419773: renameatx_np with RENAME_SWAP returns success but destroys the destination file. On any FSKit volume a RENAME_SWAP is performed as an ordinary clobbering rename: rc=0, but the destination's contents are silently lost instead of exchanged. The module cannot refuse it because renameItem receives no flags; a swap and a plain overwriting rename look identical. (RENAME_EXCL works correctly.) FB24419825: a negative lookup is cached permanently. Once anything gets ENOENT for a name on an FSKit volume, the kernel serves that ENOENT for the life of the vnode. If the file is created later (for example by another machine on a network volume), it stays unopenable by that name indefinitely, while ls of the same directory lists it. There is no API through which a module can report that a name now exists. FB24419858: a data-cache grant from openItem can be applied after the module has already invalidated. The grant in FSOpenItemResult is applied asynchronously after the module's reply, and an invalidation issued in that window succeeds (setCacheState returns no error) and is then overwritten by the stale grant. The result is a kernel cache no future event will invalidate; readers see stale data. FB24419870: synchronize(flags:) is never called on a URL-backed volume. fsync(2), fcntl(F_FULLFSYNC), fcntl(F_BARRIERFSYNC) and sync(8) all return success with zero calls reaching the module, so durability is reported and never established. A packet capture of the same SMB share shows five SMB2 FLUSH requests through Apple's smbfs and zero through an FSKit module. FB24419894: FSItemSetAttributesRequest.consumedAttributes is never observed, and wasAttributeConsumed(.changeTime) answers about the wrong attribute. Consuming everything and consuming nothing are indistinguishable to the caller (chmod returns 0 either way), even though the setAttributes documentation says the upper layers will detect unsupported attributes. Separately, wasAttributeConsumed answers YES for changeTime when only accessTime was consumed, and never answers correctly about changeTime itself; this part reproduces by constructing the request directly, no file system needed. FB24419911: restrictsOwnershipChanges = true does not reject non-superuser chown. The property is documented as "the volume rejects a chown(2) from anyone other than the superuser", but on an -o owners mount a non-root chgrp is delivered to the module's setAttributes anyway, so every module has to enforce the policy itself. FB24419932: a failed activate wedges the resource URL. After a module's activate throws once, every later mount of the same URL string fails with "Resource busy" (fskitd logs "Can't start new task, resource state is 5"), while the same volume under a different URL spelling mounts fine. For a network module the ordinary trigger is one wrong password. Recovery requires killing both fskitd and the extension process. FB24419964: enumeration cannot report extended-attribute presence. FSItem.Attributes has no per-item "has xattrs" field, so one cold ls -l of a 500-entry directory costs about 2,000 FSKit boundary crossings: an xattr call per entry plus a "._name" AppleDouble sidecar lookup per entry, and each of those ENOENTs is then pinned by FB24419825. Suggestion: a per-entry hasExtendedAttributes flag so getattrlistbulk can be satisfied from the enumeration. FB24419974: no byte-range lock operations. flock(2) and fcntl(2) locks on an FSKit mount stay kernel-local and never reach the module, so advisory locks cannot coordinate between clients of a network file system. Suggestion: an optional lock-operations handler. FB24419979: no ACL or security descriptor operations. ls -le, chmod +a, acl_get_file(3) and cp -p with ACLs cannot work on any FSKit volume; a network server's real ACLs are invisible behind synthesized mode bits. The nearest surface, FSVolumeAccessCheckHandler, can only be asked yes/no questions about a descriptor the module has no way to provide. Suggestion: an optional ACL-operations protocol. If any of these are biting you too, duplicate feedbacks referencing the FB numbers above genuinely help with prioritization.
5
1
642
3w
NFSv4.1: racing open/unlink/recreate of the same filename can leave processes unkillable
I've been building an SMB client and hoping to ship it as an FSKit module, but because of some blocking issues I decided to serve it over NFS instead. Unfortunately, I've run into another blocker, which I'll share in case others are seeing the same behavior. While testing against an NFSv4.1 server (stock Linux nfsd), I ran into a situation where processes on the Mac end up permanently blocked inside the NFS client, and I wanted to share it in case others hit the same thing. Filed as FB24538163. The trigger is several processes concurrently opening, unlinking and recreating the same filenames in one directory. Something like ten shell loops each doing cat, rm, and echo > over the same five names will do it. When it happens: The stuck processes ignore SIGKILL and sit in state U indefinitely (I have had them survive more than eight hours). umount -f on the mount blocks the same way, so the mount cannot be cleared either. Only a reboot recovers. The rest of the system stays responsive. Two details that may help narrow it down: It really is the name collision, not the load. The same loops using distinct filenames per process, at the same traffic volume and latency, ran clean for twenty minutes, while the same-name version wedged every time. Adding 8 ms of reply latency made it roughly ten times more frequent. It happens on a soft mount (timeo=100,retrycnt=3), where I would have expected EIO after the retry budget instead of an indefinite wait. Spindumps show the blocked threads inside nfs_vnop_open / nfs4_vnop_create, down through nfs4_open_rpc_internal into nfs_node_set_busy_helper, with one thread typically waiting on a write RPC reply (nfs_wait_reply) inside that same open path. A self-contained repro script is attached to the Feedback. The script only needs any NFSv4.1 server to point at, and its header has a one-line Docker command that produces one. This looks related to what the FUSE-T project has reported (macos-fuse-t/fuse-t issues 112 and 45), since FUSE-T rides the same client. If anyone knows a mount option or usage pattern that avoids the wait, or can confirm seeing this elsewhere, I would love to hear it.
0
2
167
3w
Disable Foveation for ImmersiveSpace?
Does anyone know how I can disable foveation for an ImmersiveSpace? I'm aware that I could use a CompositorLayer and my own Metal rendering to control foveation, but I'm hoping that I can configure an existing/underlying LayerRenderer (or similar) to disable it for an immersive scene. Or if there's another approach I should be taking, any pointers are appreciated. Thank you!
1
1
664
Dec ’24
Should the MV-HEVC Encoder Support Multiple Passes?
As a straightforward example, I've taken Apple's MV-HEVC sample project and added two lines. First, after the AVAssetWriterInput is created: frameInput.performsMultiPassEncodingIfSupported = true Second, after the call to multiviewWriter.startWriting(): print("canPerformMultiplePasses: \(frameInput.canPerformMultiplePasses)") Which prints true. This leads me to believe that the first encoding pass should proceed as-normal (even though I haven't handled the logic for the completion of the first pass, etc.). However, I receive this error when the code attempts to appendTaggedBuffers to the AVAssetWriterInputTaggedPixelBufferGroupAdaptor: Fatal error: Failed to append tagged buffers to multiview output Am I missing a step? Or is the multi-pass encoding only supported for standard sample/pixel buffers (and not tagged buffers)?
1
3
868
Oct ’24
Example Spatial/Immersive Video Player
I know there have been a lot of questions about playing back spatial/immersive/MV-HEVC video content on the Vision Pro. Today, I released an example player on GitHub that might answer some questions. Of course, without official documentation on some of these formats, it could be that Apple will eventually do something a little different. We'll just have to wait. In the meantime: https://github.com/mikeswanson/SpatialPlayer
1
0
1.1k
Mar ’24
MV-HEVC/Spatial Video Reading and Rendering Articles
I don't know when these were posted, but I noticed them in the AVFoundation documentation last night. There have been a lot of questions about working with this format, and these are useful. They also include code samples. Reading multiview 3D video files: https://developer.apple.com/documentation/avfoundation/media_reading_and_writing/reading_multiview_3d_video_files Converting side-by-side 3D video to multiview HEVC: https://developer.apple.com/documentation/avfoundation/media_reading_and_writing/converting_side-by-side_3d_video_to_multiview_hevc
2
0
1.6k
Feb ’24
Fisheye Projection
Does anyone have any knowledge or experience with Apple's fisheye projection type? I'm guessing that it's as the name implies: a circular capture in a square frame (encoded in MV-HEVC) that is de-warped during playback. It'd be nice to be able to experiment with this format without guessing/speculating on what to produce.
2
7
1.2k
Feb ’24
MV-HEVC/Spatial Decoder/Encoder Requirements
What are the Mac hardware and software requirements to decode and encode MV-HEVC video with AVFoundation? Many of the new MV-HEVC-related keys require macOS 14.0+, so I'm guessing that macOS Sonoma or later is required on the software side. What about processor architectures? I can read an MV-HEVC source on my Apple Silicon M1. But when I run the same code on my Intel Mac mini (2018) running Sonoma 14.3, AVAssetReader's startReading() returns false. Similarly, when I try to create an AVAssetWriterInput with MV-HEVC output settings, I receive: -[AVAssetWriterInput initWithMediaType:outputSettings:sourceFormatHint:] Compression property MVHEVCVideoLayerIDs is not supported for video codec type hvc1' Is this because Intel-based Macs don't support MV-HEVC? Or am I missing something else?
0
0
1k
Jan ’24
How Can I Access The Secondary MV-HEVC Frame
I’m working with the Spatial Video related APIs in AVFoundation, and while I can create an AVAssetReader that reads an AVAssetTrack that reports a .containsStereoMultiviewVideo media characteristic (on a spatial video recorded by an iPhone 15 Pro), the documentation doesn’t make it clear how I can obtain the secondary video frame from that track. Does anyone know where to look? I've scoured the forums, documentation, and other resources, and I've had no luck. Thanks!
4
0
1.4k
Jan ’24
Ten FSKit issues found building a network file system module (all filed with minimal repros)
While building an SMB 2/3 client as an FSKit file system module (FSUnaryFileSystem + FSVolume, the macOS 27 Handler protocols), I ran into a number of framework-level issues. I have filed each one with a title starting "FSKit:" so they are easy to find, and every report has a minimal reproduction attached: a small in-memory FSKit module (no network, no disk, no cache of its own), so none of them depend on SMB. All were measured on macOS 27.0 (26A5406e and 26A5416b) with Xcode 27.0 beta 5. Summaries below in case anyone else is hitting these. FB24419773: renameatx_np with RENAME_SWAP returns success but destroys the destination file. On any FSKit volume a RENAME_SWAP is performed as an ordinary clobbering rename: rc=0, but the destination's contents are silently lost instead of exchanged. The module cannot refuse it because renameItem receives no flags; a swap and a plain overwriting rename look identical. (RENAME_EXCL works correctly.) FB24419825: a negative lookup is cached permanently. Once anything gets ENOENT for a name on an FSKit volume, the kernel serves that ENOENT for the life of the vnode. If the file is created later (for example by another machine on a network volume), it stays unopenable by that name indefinitely, while ls of the same directory lists it. There is no API through which a module can report that a name now exists. FB24419858: a data-cache grant from openItem can be applied after the module has already invalidated. The grant in FSOpenItemResult is applied asynchronously after the module's reply, and an invalidation issued in that window succeeds (setCacheState returns no error) and is then overwritten by the stale grant. The result is a kernel cache no future event will invalidate; readers see stale data. FB24419870: synchronize(flags:) is never called on a URL-backed volume. fsync(2), fcntl(F_FULLFSYNC), fcntl(F_BARRIERFSYNC) and sync(8) all return success with zero calls reaching the module, so durability is reported and never established. A packet capture of the same SMB share shows five SMB2 FLUSH requests through Apple's smbfs and zero through an FSKit module. FB24419894: FSItemSetAttributesRequest.consumedAttributes is never observed, and wasAttributeConsumed(.changeTime) answers about the wrong attribute. Consuming everything and consuming nothing are indistinguishable to the caller (chmod returns 0 either way), even though the setAttributes documentation says the upper layers will detect unsupported attributes. Separately, wasAttributeConsumed answers YES for changeTime when only accessTime was consumed, and never answers correctly about changeTime itself; this part reproduces by constructing the request directly, no file system needed. FB24419911: restrictsOwnershipChanges = true does not reject non-superuser chown. The property is documented as "the volume rejects a chown(2) from anyone other than the superuser", but on an -o owners mount a non-root chgrp is delivered to the module's setAttributes anyway, so every module has to enforce the policy itself. FB24419932: a failed activate wedges the resource URL. After a module's activate throws once, every later mount of the same URL string fails with "Resource busy" (fskitd logs "Can't start new task, resource state is 5"), while the same volume under a different URL spelling mounts fine. For a network module the ordinary trigger is one wrong password. Recovery requires killing both fskitd and the extension process. FB24419964: enumeration cannot report extended-attribute presence. FSItem.Attributes has no per-item "has xattrs" field, so one cold ls -l of a 500-entry directory costs about 2,000 FSKit boundary crossings: an xattr call per entry plus a "._name" AppleDouble sidecar lookup per entry, and each of those ENOENTs is then pinned by FB24419825. Suggestion: a per-entry hasExtendedAttributes flag so getattrlistbulk can be satisfied from the enumeration. FB24419974: no byte-range lock operations. flock(2) and fcntl(2) locks on an FSKit mount stay kernel-local and never reach the module, so advisory locks cannot coordinate between clients of a network file system. Suggestion: an optional lock-operations handler. FB24419979: no ACL or security descriptor operations. ls -le, chmod +a, acl_get_file(3) and cp -p with ACLs cannot work on any FSKit volume; a network server's real ACLs are invisible behind synthesized mode bits. The nearest surface, FSVolumeAccessCheckHandler, can only be asked yes/no questions about a descriptor the module has no way to provide. Suggestion: an optional ACL-operations protocol. If any of these are biting you too, duplicate feedbacks referencing the FB numbers above genuinely help with prioritization.
Replies
5
Boosts
1
Views
642
Activity
3w
NFSv4.1: racing open/unlink/recreate of the same filename can leave processes unkillable
I've been building an SMB client and hoping to ship it as an FSKit module, but because of some blocking issues I decided to serve it over NFS instead. Unfortunately, I've run into another blocker, which I'll share in case others are seeing the same behavior. While testing against an NFSv4.1 server (stock Linux nfsd), I ran into a situation where processes on the Mac end up permanently blocked inside the NFS client, and I wanted to share it in case others hit the same thing. Filed as FB24538163. The trigger is several processes concurrently opening, unlinking and recreating the same filenames in one directory. Something like ten shell loops each doing cat, rm, and echo > over the same five names will do it. When it happens: The stuck processes ignore SIGKILL and sit in state U indefinitely (I have had them survive more than eight hours). umount -f on the mount blocks the same way, so the mount cannot be cleared either. Only a reboot recovers. The rest of the system stays responsive. Two details that may help narrow it down: It really is the name collision, not the load. The same loops using distinct filenames per process, at the same traffic volume and latency, ran clean for twenty minutes, while the same-name version wedged every time. Adding 8 ms of reply latency made it roughly ten times more frequent. It happens on a soft mount (timeo=100,retrycnt=3), where I would have expected EIO after the retry budget instead of an indefinite wait. Spindumps show the blocked threads inside nfs_vnop_open / nfs4_vnop_create, down through nfs4_open_rpc_internal into nfs_node_set_busy_helper, with one thread typically waiting on a write RPC reply (nfs_wait_reply) inside that same open path. A self-contained repro script is attached to the Feedback. The script only needs any NFSv4.1 server to point at, and its header has a one-line Docker command that produces one. This looks related to what the FUSE-T project has reported (macos-fuse-t/fuse-t issues 112 and 45), since FUSE-T rides the same client. If anyone knows a mount option or usage pattern that avoids the wait, or can confirm seeing this elsewhere, I would love to hear it.
Replies
0
Boosts
2
Views
167
Activity
3w
Disable Foveation for ImmersiveSpace?
Does anyone know how I can disable foveation for an ImmersiveSpace? I'm aware that I could use a CompositorLayer and my own Metal rendering to control foveation, but I'm hoping that I can configure an existing/underlying LayerRenderer (or similar) to disable it for an immersive scene. Or if there's another approach I should be taking, any pointers are appreciated. Thank you!
Replies
1
Boosts
1
Views
664
Activity
Dec ’24
Should the MV-HEVC Encoder Support Multiple Passes?
As a straightforward example, I've taken Apple's MV-HEVC sample project and added two lines. First, after the AVAssetWriterInput is created: frameInput.performsMultiPassEncodingIfSupported = true Second, after the call to multiviewWriter.startWriting(): print("canPerformMultiplePasses: \(frameInput.canPerformMultiplePasses)") Which prints true. This leads me to believe that the first encoding pass should proceed as-normal (even though I haven't handled the logic for the completion of the first pass, etc.). However, I receive this error when the code attempts to appendTaggedBuffers to the AVAssetWriterInputTaggedPixelBufferGroupAdaptor: Fatal error: Failed to append tagged buffers to multiview output Am I missing a step? Or is the multi-pass encoding only supported for standard sample/pixel buffers (and not tagged buffers)?
Replies
1
Boosts
3
Views
868
Activity
Oct ’24
Example Spatial/Immersive Video Player
I know there have been a lot of questions about playing back spatial/immersive/MV-HEVC video content on the Vision Pro. Today, I released an example player on GitHub that might answer some questions. Of course, without official documentation on some of these formats, it could be that Apple will eventually do something a little different. We'll just have to wait. In the meantime: https://github.com/mikeswanson/SpatialPlayer
Replies
1
Boosts
0
Views
1.1k
Activity
Mar ’24
MV-HEVC/Spatial Video Reading and Rendering Articles
I don't know when these were posted, but I noticed them in the AVFoundation documentation last night. There have been a lot of questions about working with this format, and these are useful. They also include code samples. Reading multiview 3D video files: https://developer.apple.com/documentation/avfoundation/media_reading_and_writing/reading_multiview_3d_video_files Converting side-by-side 3D video to multiview HEVC: https://developer.apple.com/documentation/avfoundation/media_reading_and_writing/converting_side-by-side_3d_video_to_multiview_hevc
Replies
2
Boosts
0
Views
1.6k
Activity
Feb ’24
Fisheye Projection
Does anyone have any knowledge or experience with Apple's fisheye projection type? I'm guessing that it's as the name implies: a circular capture in a square frame (encoded in MV-HEVC) that is de-warped during playback. It'd be nice to be able to experiment with this format without guessing/speculating on what to produce.
Replies
2
Boosts
7
Views
1.2k
Activity
Feb ’24
MV-HEVC/Spatial Decoder/Encoder Requirements
What are the Mac hardware and software requirements to decode and encode MV-HEVC video with AVFoundation? Many of the new MV-HEVC-related keys require macOS 14.0+, so I'm guessing that macOS Sonoma or later is required on the software side. What about processor architectures? I can read an MV-HEVC source on my Apple Silicon M1. But when I run the same code on my Intel Mac mini (2018) running Sonoma 14.3, AVAssetReader's startReading() returns false. Similarly, when I try to create an AVAssetWriterInput with MV-HEVC output settings, I receive: -[AVAssetWriterInput initWithMediaType:outputSettings:sourceFormatHint:] Compression property MVHEVCVideoLayerIDs is not supported for video codec type hvc1' Is this because Intel-based Macs don't support MV-HEVC? Or am I missing something else?
Replies
0
Boosts
0
Views
1k
Activity
Jan ’24
How Can I Access The Secondary MV-HEVC Frame
I’m working with the Spatial Video related APIs in AVFoundation, and while I can create an AVAssetReader that reads an AVAssetTrack that reports a .containsStereoMultiviewVideo media characteristic (on a spatial video recorded by an iPhone 15 Pro), the documentation doesn’t make it clear how I can obtain the secondary video frame from that track. Does anyone know where to look? I've scoured the forums, documentation, and other resources, and I've had no luck. Thanks!
Replies
4
Boosts
0
Views
1.4k
Activity
Jan ’24